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Abstract 

Guidance  laws  are  developed  to  optimally  position  a  relay  Micro-UAV  (MAV)  to 
provide  an  operator  at  the  base  with  real-time  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  by  relaying  communication  and  video  signals  when  the  rover 
MAV  performing  the  ISR  mission  is  out  of  radio  contact  range  with  the  base.  The  ISR 
system  is  comprised  of  two  MAVs,  the  Relay  and  the  Rover,  and  a  Base.  The  Relay 
strives  to  minimize  the  radio  frequency  (RF)  power  required  for  maintaining 
communications,  while  the  Rover  performs  the  ISR  mission,  which  may  maximize  the 
required  RF  power.  The  optimal  control  of  the  Relay  MAV  entails  the  solution  of  a 
differential  game.  Suboptimal  solutions  are  also  analyzed  to  gain  insight  into  the  solution 
of  the  differential  game.  One  suboptimal  approach  investigated  envisages  the  Rover  to 
momentarily  remain  stationary  and  solves  for  the  optimal  path  for  the  Relay  to  minimize 
the  RF  power  requirement  during  the  planning  horizon:  the  one  -  sided  optimal  control 
problem  is  solved  in  closed  form.  Another  suboptimal  approach  is  based  upon  the 
geometry  of  the  system:  The  midpoint  between  the  Rover  and  the  Base  is  the  location 
which  minimizes  the  RF  power  required,  so  the  Relay  heads  toward  that  point — assuming 
that  the  Rover  is  stationary.  At  the  same  time,  to  maximize  the  required  RF  power,  the 
Rover  runs  away  from  the  Relay.  In  this  work  the  differential  game  is  fully  analyzed.  The 
geometry  based  suboptimal  solution  is  shown  to  be  the  optimal  solution  in  the  endgame. 
Isaac’s  method  is  then  applied  to  obtain  the  optimal  trajectories  in  the  differential  game. 
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A  RELAY-ROVER  DIFFERENTIAL  GAME 


I.  Introduction 


1.1.  Background 

Unmanned  Aerial  Vehicles  (UAVs)  are  prevalent  in  current  military  operations. 
UAVs  vary  in  size  and  mission.  While  some  UAVs  are  the  same  size  as  aircraft,  others 
are  man-portable  and  can  be  carried  in  a  backpack.  These  man -portable  Micro-UAVs 
(MAVs)  utilized  by  small  tactical  units  are  not  supported  by  satellite  communications  and 
use  radio  frequency  (RF)  modems.  High  frequency  radio  communications  are  range 
limited.  The  MAVs  considered  in  this  paper  are  utilized  for  Intelligence,  Surveillance  and 
Reconnaissance  (ISR)  and  will  therefore  be  referred  to  as  ISR  MAVs  or  as  Rovers  [1]. 

The  Base  may  lose  communication  (and  controllability)  with  deployed  ISR 
MAVs/Rovers  if  the  Rovers  stray  far  away.  In  this  thesis  guidance  laws  are  developed  to 
optimally  position  a  Relay  MAV  to  provide  the  operator  at  the  Base  with  real-time  ISR 
by  relaying  communication  and  sensor  data  while  allowing  for  extended  range  Rover 
operations.  The  Relay-Rover  interaction  is  modeled  as  a  differential  game  whose  solution 
yields  the  optimal  Relay  strategy. 

1.2.  Research  Objectives 

The  ISR  mission  is  considered  where  a  rover  MAV  is  controlled  by  an  operator 
and  tasked  to  fly  to  locations  where  targets  need  to  be  inspected.  Since  the  rover  MAV 
might  get  out  of  radio  communication  range  with  the  base  station  where  the  operator  is 
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located,  it  is  envisaged  that  a  relay  MAV  will  be  interposed  between  the  rover  and  the 
base  station,  so  that  connectivity  will  be  maintained. 

In  order  to  reduce  the  workload  of  the  operator,  the  relay  MAV  will  be 
autonomous.  In  other  words,  a  guidance  law  for  the  relay  MAV  will  be  developed  such 
that  the  relay  MAV  will  automatically  position  itself  between  the  rover  and  the  base,  with 
a  view  to  reducing  the  RF  power  required  for  communications  -  this,  despite  the 
maneuvers  of  the  rover.  The  worst  case  scenario  is  considered,  where  the  rover  is  “giving 
a  hard  time”  to  the  relay  MAV.  This  calls  for  a  min-max  optimal  control  formulation,  a 
differential  game  formulation. 
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II.  Literature  Review 


To  better  appreciate  the  nature  of  the  thesis  research,  it  is  necessary  to  first 
discuss  previous  relevant  works.  Recent  studies  have  produced  two  general  designs  for  a 
reliable  and  robust  communication  network  utilizing  mobile  communication  nodes.  The  first 
design  consists  of  one  or  more  mobile  communication  nodes  which  form  a  single  chain  to 
relay  information  between  the  source  and  the  destination.  This  is  referred  to  as  a  “single -flow 
network.”  Many  designs  for  single-flow  networks  use  a  fixed  source  and  fixed  destination, 
though  they  do  not  discount  the  possibility  of  a  mobile  destination  (Dixon  and  Frew,  2007; 
Goldenberg  et  al.,  2004).  The  second  design  consists  of  multiple  mobile  communication 
nodes  which  form  a  “mesh-like  network”.  This  configuration  adds  fault-tolerance  for  a  more 
robust  network  (Basu  and  Redi,  2004;  Floreano  et  al.,  2007).  However,  a  “mesh-like 
network”  would  be  ill  suited  for  the  envisioned  ISR  and  engagement  system  due  to  desired 
unit  covertness  while  engaging  a  high  value  target.  In  this  respect,  Brown  et  al.  have 
developed  the  Ad-hoc  UAV  Ground  Network  (AUGNet)  test  bed,  showing  the  practicality  of 
UAV-based  mobile  communication  nodes  using  IEEE  802.1  lb  wireless  routers  (Brown  et  al., 
2004).  The  proposed  ISR  and  engagement  system  may  have  a  network  design  similar  to 
AUGNet  but  the  Relay  must  still  have  an  optimal  mobility  control  law  in  order  optimize 
network  communications. 

Dixon  and  Frew  have  utilized  the  AUGNet  system  with  an  extremum  seeking 
controller  to  study  cooperative  electronic  chaining  while  maximizing  the  signal -to-noise  ratio 
between  the  nodes  of  the  multi-hop  network  (Dixon  and  Frew,  2007).  Goldenberg  et  al.  have 
shown  that  communication  nodes  should  be  evenly  spaced  on  the  line  between  the  source  and 
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destination  in  order  to  minimize  the  energy  cost  of  communicating  between  the  two 
(Goldenberg  et  al.,  2004). 

This  line  of  research  was  initiated  by  Lt.  John  H.  Hansen  in  2008,  in  his  thesis 
‘Optimal  Guidance  of  A  Relay  MAV  for  ISR  Support  Beyond  Line-Of-Sight’[7].  In  this 
thesis,  the  basic  system  definition  from  his  work  will  be  introduced  again  and  used  for 
further  research.  Additional  work,  including  limited  experimentation  and  hardware 
testing  was  done  by  the  students  of  Dr.  David  R.  Jacques. 

Building  on  these  foundations,  in  this  thesis  the  underlying  differential  game 
theory  is  further  developed. 


II.l.  System  Definition 

It  is  assumed  that  the  rElay  (E)  MAV  is  cognizant  of  the  rOver’s  (O) 
instantaneous  position  and,  obviously,  own  ship  position.  As  far  as  the  RF  power 
requirements  are  concerned,  this  is  determined  by  their  distance  from  the  Base  (B)  and 
the  rOver-rElay  separation.  Thus,  the  state  is  the  distance  rE  of  the  rElay  from  the  Base, 
the  distance  r0  of  the  rOver  to  the  Base,  and  the  angle  9  included  between  the  radials 
from  the  Base  to  the  rElay  and  the  rOver.  This  angle  is  measured  clockwise.  The  MAVs 
have  simple  motion.  The  control  for  each  MAV  is  its  relative  heading  angle  measured 
clock-wise  from  its  radial  from  the  Base.  Figure  1  provides  a  visualization  of  the 
kinematics.  The  differential  equations  of  motion  are 


4  =  VE  cos  (p  ,r£(0)  =  r£o 

rQ=V0  cosy/  5  ro (0)  =  roa 

0  =  -^-V0smy/  -±VEsm(p  ,  9(0)  =  90,  0  <t<T 


(1) 
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T  is  the  planning  horizon  utilized  by  the  control  algorithm.  The  cost  functional  is 
indicative  of  the  RF  power  required  and  is  the  time  averaged  sum  of  the  squares  of  the 
distance  between  the  rElay  and  the  rOver  and  between  the  rElay  and  the  Base: 

y  =  j^(Ed\t)  +  BE2(t))jdt 


Figure  1:  Schematic  of  Relay  System 

The  points  E,  B  and  O  in  1R.2  represent  the  positions  of  the  rElay,  Base  and  rOver 
respectively.  These  three  points  form  a  triangle  which  can  be  utilized  to  calculate  the 

distance  EO  by  the  law  of  cosines. 

EO  (0  =  rl  +  r^  -  2 rEr0  cos  6 

Hence  the  cost  functional  is 
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(2) 


y  =  £  ( 2r£2  +  r-  -  2 rEr0  cos  0)dt 

The  objective  is  to  minimize  the  average  RF  power  required  for  maintaining 
communications.  The  control  available  to  accomplish  this  task  is  limited  to  setting  the 
course  angle  <p  of  the  rElay,  while  the  rOver  performs  the  ISR  mission:  in  a  worst  case 
scenario,  one  might  assume  that  the  rOver  is  working  to  maximize  the  cost  functional. 
The  optimization  problem  is  then  a  differential  game  [2]  where  the  rElay’ s  control  is  its 
relative  heading  cp  and  the  rOver’s  control  is  its  relative  heading  i//. 

The  system  is  analyzed  by  first  non-dimensionalizing  the  states  and  the 
parameters.  The  velocities  are  scaled  by  the  velocity  of  the  rElay  (Ve),  yielding  a  non- 
dimensional  speed  ratio  a .  The  distances  are  scaled  by  d,  where  d  is  a  characteristic 

r  rn  V  V 

length,  say  d -VET .  Set  rE:=— ,  rG:=— ,  t  :=  —  •?,  T  :=  —  -T  and  the  speed  ratio 

d  d  d  d 

V 

a  =  — .  Using  these  non-dimensional  variables  and  parameters,  the  two  sided 

^ E 

optimization  problem  now  becomes 

min  max  y  =  £  ( 2 rE  +  -  2 rEr0  cos  dt 

s.t. 

rE=cos<p  ,rE(0)  =  rEo  (3) 

r0  =  a  cos^  ,ro(0)  =  rOo 

9  =  -^asinif/--^sm(p  ,  6{0)  =  60,  0<t<T 

The  problem  parameters  are  the  speed  ratio  a  >  0  and  the  planning  horizon  T  >  0 . 
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Since  the  optimal  control  problem  only  makes  sense  if  the  rOver  is  closer  to  the 
rElay  than  to  the  Base  (B),  the  following  must  hold. 

rE  <  2r0cos9 . 

Thus,  since  the  problem  is  symmetric  about  the  0  -  Oaxis,  the  state  space  is 

|( ,  r0 , 6»)|  0  <  6»  <  ^ ,  °  <  r£  <  2r0  cos  6>} 

To  solve  the  differential  game,  the  Hamiltonian  is  introduced  in  eq.  (4), 

3C  =  -2 rE  -  +  2 rErQ  cos  0  +  Xr<  cos  cp  +  \ja cos  y/  +  (-^  a  sin  y/  -  sin  cp  )  (4) 

where  Ar  and  A0  are  the  system  co-states. 

According  to  the  Pontryagin  Maximum  Principle  (PMP)  [3],  the  differential 
equations  for  the  co- states  are 

4=4r£-2rocos0-4^  ,  \(T)  =  0 

rE 

i,o  =  2 r0  -  2 rE  cos  9  +  a  9  ^  (T)  =  0  >  (5) 

ro 

Ae  =  2 rEr0  sin  9  ,  Ae(T)  =  0 

and  the  optimality  condition  is  given  by  max  min  3C ,  namely 

<P  ¥ 


or, 


7 


(7) 


sin  <p  =  ■ 


Aa 


i/\V+V 


cos^  = 


rA 


Vv*2+v 


<33C  ,  .  Am  cos  if/  n 

- =  -A.  a sm if/ +  — - - =  0 

di//  0  ra 


tan  if/*  = 


An 
^ _  _ v_ 


K/o 


or, 


sm  y/  = 


An 


r—n — ; - 7.  coW  =■  r~0 — - 

4ro  K0  +  yjr0  ArJ  +A( 


ro\, 


The  second-order  sufficiency  condition  for  (p  is 


d25C  .  Ag  sin  cp 

— J  =  -Ar  cos cp H — - -<0 


dcp 


and  inserting  the  expression  for  iff'  from  (7)  yields 


rEAr  Y 

An+~ - ^-<0 

K 

=>  AM)  <  0  V  0 <t<T 


Similarly, 


d2%  ,  AnCcsinif/ 

=  -A  a  cos  if/  — - >  0 

dif/~  r0 


and  inserting  the  expression  for  1//*  from  (7)  yields 


(8) 
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(9) 


/L 


X,Q 


<0 


A0(t)<O  v  o<t<r 


The  expressions  for  (p*  and  <//*  given  in  Equations  (7)  and  (8)  can  also  be  used  to 
rewrite  the  state  and  co-state  equations  only  in  terms  of  the  states  and  co-states.  A 
standard,  albeit  nonlinear,  Two-Point  Boundary  Value  Problem  (TPBVP)  on  the  interval  1 
=  [0,  T]  is  obtained: 


rE  = 


4o 


ro  = 


\]Ar/:  rE  +  At 


-«Vo 


0  = 


\jAlV0+Ae 

4 _ „ 

rE^KErE+K  ro^I0 


aAn 


K  ~  4r£  -  2 r0  cos  6  +  - 


1  =  2  rn  -  2 rF  cos  0  -  - 


4 


'\lKrE  +  4 


’  rE(° )  =  rE0 

>  ro(°)  =  roQ 

,  m=o0 

?,  400  =  0 


aAg 


l^K/o+Ai 


a  4(0  =  0 


4  =  2r£r0  sin  6* 


,  4(T)  =  0,  0<t<T 


(10) 


Note:  if  0T  ^  0,  the  costate  4(0  <  0  V0  <  t<T ;  also  +  4  *  0’ 

sfipi  +  A  *0V0<t<T. 
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III.  Methodology 


III.l.  The  End-Game 

At  time  t=T  where  the  co-states  vanish,  and  since  the  control  variables  do  not 
explicitly  feature  in  the  cost  functional,  it  is  impossible  to  calculate  the  terminal  controls 
by  applying  the  PMP  and  maximizing  and  minimizing  the  Hamiltonian  (4);  obviously, 

equations  (10)  don’t  apply  because  ^22r£2  +  22  =  0 .  The  end  game  requires  special 

attention. 

Since  the  end  state  is  free  -  that’s  why  the  co-states  vanish  at  t=T  -  the  rElay’s 
and  rOver’s  optimal  strategies  at  t=T  are  myopic.  Thus,  from  first  principles,  the  rElay 
would  want  the  integrand  in  the  cost  functional 

L(rE ,  rQ ,  9)  =  2rE2  +  rQ 2  -2 rEr0  cos  6 

to  be  minimized.  Similarly,  the  rOver  would  want  the  integrand  in  the  cost  functional  to 
be  maximized.  However  the  control  variables  (p  and  t//  do  not  directly  feature  in  L. 
Hence  the  rElay  minimizes  and  the  rOver  maximizes  the  temporal  derivative  of  L, 
evaluated  at  t=T : 


d_ 

dt 


L  =  2 


2 rE  cos  cp  +  ar0  cos  y/  -  r0  cos  <pcos6-  arE  cos  \p  cos  6  +  rEr0  sin  6 


f  1  . 

a — sin  \p 

V  ro 


=  2|(2r£-r0cos^)cos^-r0sin^  sin^  +  a  (r0 -r£cos#)cos^  +  r£sin#siny/r]j 


1  ■  1 

— sin  <p 

rE  J 


The  rElay  and  rOver  solve  the  respective  static  optimization  problems 

min  [(2rE~rO  cosO)cos(p-r0  sin^  sin^>]  (11) 
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and 


max[(r0  -rEcosO)cosy/  +  rEsm6smy/^  (12) 

Concerning  eq.  (11):  minimizing  in  cp  the  derivative  of  the  integrand  at  time  t=T  yields 
the  rElay’s  optimal  terminal  control 


cos  (<p*  (T))  = 


roT  cos  ^  rE, 


■\j^rET  rOT  ^rET  roT  cos 


,  Sin(p*(r))  = 


>0,  sin  0, 


\J^rET  rOT  ^rETrOT  COS@T 


provided  that  rE  ,  rQ  and  0T  are  s.t.  the  following  is  not  the  case:  rEf  =  —  r0/  and  0T  =0. 


Here,  (r^  ,  r0/ ,  0T )  denote  the  terminal  state  at  time  t=T. 


When  0T  =0,  eq.  (11)  yields 


<P(T)  = 


0  if  rF  <  —  r,. 

lLj  ^  '-'T 

•r  1 

n  if  rF  >—rn 

Ljj  ^  '-'J 


An  inspection  of  Fig.  2  tells  us  that  in  the  end  game,  at  time  t=T,  the  rElay  heads 


toward  the  midpoint  M  of  the  segment  BO. 
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V\T) 


jO  if  r0r  >rEr 

V  if  ^  <  % 
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An  inspection  of  Fig.  2  tells  us  that  in  the  end  game,  at  time  t=T,  the  rOver  runs 
away  from  the  rElay. 

III.2.  Special  Case  -  Optimal  Control  Problem 

We  are  interested  in  the  solution  of  the  zero-sum  differential  game  which,  loosely 
speaking,  is  a  two-sided  optimal  control  problem.  When  the  rOver  is  stationary  the  speed 
ratio  parameter  a  —  0 ,  and  the  rElay  is  faced  with  a  simpler  optimal  control  problem. 
Subscribing  to  Polya’s  [4]  dictum:  “If  you  cannot  solve  the  problem  on  hand,  there  surely 
is  a  simplified  version  of  the  problem  which  you  also  don’t  know  how  to  solve  -  solve 
that  problem  first”,  we  consider  the  case  where  the  rOver  is  stationary  and  we  first  obtain 
the  closed-form  solution  of  the  optimal  control  problem. 

The  optimal  control  problem  considered  is 

min  £  (rE  ~  rEro  cos  dt 
s.t. 

4  =V£cos  (p  ,  rE  (0)  =  vEq 
6  =  -^sin#?  ,  0(0)  =  0O,  0  <  t  <  T 

This  is  equivalent  to  setting  a=0  in  the  differential  game  (3)  the  parameter. 
Without  loss  of  generality  we  use  the  characteristic  length  d  —  rQ,  that  is,  using  non 
dimensional  variables,  the  rOver  is  stationary  at  rQ= 1  and,  the  optimal  control  problem  is 
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min  Jo  ( rE  ~  rE  cos  0}  dt 
s.t. 

rE  —  cos (p  ,  rE(0)  =  rEQ 
d  =  —-)-sin(p,  0(0)  =  0o,  0<t<T 


The  state  space  is  the  half  disc  of  unit  radius  centered  at  (1,0), 


(rE,0)\O<0  <  —  ,  O<r£<2cos0 


The  control  0  <  (p  <  n  and  the  problem  parameter  is  T  {t  =  ppT  j 


III.2.1.  Analysis 


In  the  optimal  control  problem  the  Hamiltonian 


9C  =  rE  cos  0  -  rE  +  Ar  cos  (p  -  Ae  j-  sin  cp 


Applying  the  Pontryagin  Maximum  Principle  (PMP)  we  obtain 


sin  cp  = 


/W+V 


cos  cp  =  ■ 


provided  the  co-states  Ae  and  A  don’t  vanish. 


The  co-states’  differential  equations  are 


Ae  sin  cp 


\=2r*-cos<9— ,A,e(T)  =  0 

rE 


An  =  rF  sin  0  , 


Ao(T)  =  0 
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The  TPBVP  for  the  optimal  control  problem  is 


A  r 

r_  E 


rE  = 


0  = 


V 


*2  2  2 

/L  r  +  /L 

r_  E  6 


Xn 


*  2  2  2 

1  r  +1 

r_  £  0 


,  0(0)  =  0O 


/l  =2  r  —  cos  0  + 


A; 


2  *2  2  *2 

r  Ja  rr  +  /L 


■v 


,  a  (r>  =  o 


4  =^sin6» 


,  UT)  =  Q,  0  <t  <T 


(13) 


Note:  if  0T  ^  0 ,  the  costate  A0(f)  <  0  VO <  t  <  T  ;  also,  +  Ag2  ^  0  VO  <  t  <  T  . 

III.2.2.  The  End-Game  in  Optimal  Control  Problem 

As  in  the  differential  game  at  time  t=T  where  the  co-states  vanish,  and  since  the 
control  variable  (p  does  not  explicitly  feature  in  the  cost  functional,  it  is  impossible  to 
calculate  the  terminal  control  by  applying  the  PMP  and  maximizing  the  Hamiltonian; 

obviously,  equations  (13)  don’t  apply  because  ^l2r£2  +A 2  =  0.  The  end  game  requires 
special  attention. 

Since  the  end  state  is  free  -  that’s  why  the  co-states  vanish  at  t=T  -  the  rElay’s 
optimal  strategy  at  t=T  is  myopic.  Thus,  from  first  principles  it  is  clear  that  the  rElay 
would  want  the  integrand  in  the  cost  functional, 

L(rE ,  6)  =  rE  —  rE  cos  0 
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to  be  minimized.  However  the  control  variable  (p  does  not  feature  in  L.  Hence  the  rElay 


minimizes  the  temporal  derivative  of  L,  evaluated  at  t=T. 


—  L-2rE  cos  (p-cos(p  cos  0  -  sin  0  sin  (p 
dt 


=  (2r£  -  cos  0)  cos  ^  -  sin  0  sin  ^ 


Minimizing  in  the  derivative  of  the  integrand  at  time  t=T  yields 


sin 


■(*>'(*•))= 

h>‘h))= 


cos^j.  -2  rE 


\J^rEr  1~  4/y^  cos  0T 
sin  0T 

^4rET2+l-4rETCos0T 


(14) 


Here  ( rE/ ,  r0j ,  0r  j  is  the  state  at  time  t=T. 

Note  that  as  in  the  differential  game,  when  0T  -  0 ,  the  optimal  terminal  rElay  control  is 


9(T)  = 


0  V  0  <  rr  <  — 


n  V  —  <  rF  <  2 
2  £ 


(15) 


As  in  the  differential  game,  an  inspection  of  Fig.  2  and  eqs.  (14)  and  (15)  tell  us 


that  in  the  end  game,  at  time  t=T,  the  rElay  heads  toward  the  mid-point  M=(  —  ,0). 


The  analysis  yielding  the  optimal  terminal  rElay  strategy  (14)  and  (15)  applies,  provided 
that  the  end  state  (rEr,0T)  is  not 
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eT  =o, 


1 

2 


III.2.3.  Retrograde  Integration:  t  <  T 

At  time  t=T  the  co-states  vanish  and  we  cannot  use  eqs.  (13);  also, 

K  2r2  +  A02  -  0  at  t  =  T.  At  the  terminal  time  t  =  T  where,  the  maximization  of  3C  does 

not  yield  the  optimal  control  according  to  the  PMP,  we  must  use  eqs.  (14)  and  (15) 
derived  from  first  principles.  Thus,  in  the  end  game  at  the  terminal  time  t=T,  the  specified 
rElay  control  in  eqs.  (13),  is  replaced  by  the  rElay  control  in  the  end  game,  namely,  eq. 
(14).  Thus,  at  time  t=T  eqs.  (13)  are  replaced  by 


We  can  now  kick  start  the  backward  integration  at  t  =  T,  from  any  end  state  in  the 


state  space,  except  the  special  end  state  rE/ 


1 

2 


,  0T  -  0,  where  the  terminal  control 


cannot  be  calculated.  Also  recall  that  the  costate  A0(t)  <  0  VO  <  t  <  T  ,  provided  that  0T 
>0.  We  are  particularly  interested  in  the  trajectories  which,  under  optimal  play,  terminate 

at  the  midpoint  M  of  the  segment  BO  where  rE(T)  =  6(T)  =  0  -  referred  to  as  the 
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‘sweet  spot”.  When  the  end  point  is  the  “sweet  spot”  0T  -  0 ,  rE  =  —  we  cannot  obtain 


the  optimal  rElay  control;  the  terminal  optimal  control  <P  \T)  cannot  be  determined  from 
an  examination  of  the  equation 


=  (2rE  ~ cos  0) cos  (p- sin  0 sirup 


=  O-cos^-O-siny? 

The  same  applies  when  higher  derivatives  of  L  are  considered.  Eq.  (14)  does  not 
apply  and  the  retrograde  integration  of  eqs.  (13)  cannot  be  “started”  at  the  special  “sweet 

spot”  tenninal  state  rE^  =^,  0T=0  using  eqs.  (16).  Yet,  as  we  shall  see  -  see,  e.g., 

Figure  4  in  the  sequel  -  it  turns  out  that  all  the  optimal  trajectories  radiate  “out”  of  the 

“sweet  spot”  M  where  rE  =  ,  0  =  0  .  In  other  words,  for  all  initial  states  ( rE> ,  0o )  in  the 


state  space  there  exists  a  planning  horizon  T(rEg,  60)  s.t.  rE(T(rEg,  60))  =  —  and 


0(n%,  0„))=o. 

Furthermore,  given  the  initial  state  (r  0o) ,  if  the  planning  horizon 

T  >  T( ,  0O) ,  the  optimal  trajectory  is  s.t.  the  “sweet  spot”  rK  =  — ,  0=0  is  reached  at 

time  T(rE  ,  0o) ,  following  which,  during  the  time  interval  T(rE  ,  0o)  <t<T,  chattering 

control  will  be  applied  and  the  rElay  will  stay  put  at  the  “  sweet  spot”.  Hence  we  back  off 
and  instead  “start”  the  retrograde  integration  from  a  family  of  end  points  located  on  the 
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circumference  of  a  small  semicircle  of  radius  0  <  e  <C  1  centered  at  the  “sweet  spot”, 


namely,  the  mid-point  M  = 


0 


of  the  segment  OB  -  as  shown  in  Fig.  3. 


Figure  3:  End  Points  Manifold 


19 


i  i  i  i  r 


,15i _ i _ i _ i _ i _ i _ 

'■0.5  0  0.5  1  1.5  2  2.5 

Figure  4:  The  Optimal  Flow  Field 

The  characteristics  are  integrated  in  retrograde  fashion  “starting”  out  at  end-points 
Et  =  (rEi ,  0T) ,  where 


.  2  f  1  V  1  / - j 

rF  =.  \s  sin  <5+  —  +  s  cos<£  =  —  Jl  +  4e  cos  t  +  As 

Et  ]]  U  J  2y 

and 

(  \ 

^sin^ 


0T  =  Atari 


1 

—  +  £  COS  C 

U 


(17) 


One  proceeds  as  follows.  One  “starts  out”  from  a  point  Er  on  the  terminal  semicircular 

manifold  shown  in  Fig.  3.  Eqs.  (17)  are  inserted  into  the  R.H.S.  of  eqs.  (16)  where  the 
optimal  control  in  the  end  game  is  used  and  equations  (16)  are  then  used  to  step  back  one 
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time  increment  to  time  T  —AT .  From  that  point  on,  having  gotten  away  from  the  end 

state,  and,  in  particular,  the  critical  end  state  rF/  =  — ,  0T  -  0 ,  eqs.  (13)  where  the  optimal 

control  specified  by  the  PMP  is  used  are  integrated  in  retrograde  fashion  V  T-AT  >  t  >  0 . 
The  integration  is  stopped  when  the  condition  r£<2cos#  is  violated,  that  is,  the 
boundary  of  the  state  space  is  reached.  The  end  points  manifold  is  the  thick  line 

(1  \ 

semicircle  of  radius  s  centered  at  — ,  0  ,  as  shown  in  Fig.  3.  The  family  of  end  points 

v2  ) 

Et  is  parameterized  by  the  angle  q  ,  0  <^<n  . 


The  rElay’s  optimal  strategy  in  the  end  game  is  in  fact  the  optimal  strategy 
throughout,  as  proven  in  Section  3.4  in  the  sequel.  Hence  the  optimal  trajectories  leading 
into  the  semicircular  terminal  manifold  are  straight  lines.  Also  note  that  if  0T  =  0 ,  that  is, 
=  0  or  £  =  n ,  the  rElay  stays  on  the  horizontal  line  BO  -  the  symmetry  axis  of  the  disc 
shaped  state  space  -  and  heads  toward  the  mid-point  M  of  the  segment  OB  . 

Consequently,  the  optimal  flow  field  consists  of  straight  line  trajectories  which 
converge  at  the  “sweet  spot”  M.  This  results  in  the  optimal  flow  field  shown  in  Fig.  4:  the 
optimal  state  feedback  strategy  is 

.  *.  cos 6 - 2rF  .  *  sin# 

cos(^  (rE,  0))=  7 — —  sm(^  (rE,  0 ))  =  — _ 

y  4 rE  + 1  -  4r£  cos  9  J- 4rE  + 1  -  4 rE  cos  6 


and  the  ensuing  optimal  straight  line  trajectories,  all  converge  to  the  “sweet  spot”  M= 
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III.3.  Analytic  Solution  of  the  Optimal  Control  Problem 

In  the  optimal  control  problem  there  are  just  two  state  variables,  rE  and  6.  It  is 
therefore  instructive  to  use  Cartesian  coordinates  (x,  y)  as  shown  below. 


Figure  5:  Trajectory  in  Cartesian  Plane 


The  dynamics  now  are 

x  =  sin  cp ,  x(0)  =  x0 
y  =  -cos  <p,  y( 0)  =  y0,  0  <  t  <  T, 

and  the  cost  functional  is 

y  =  2 J  (x2  +  y2  -  x)  clt 
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The  state  space  is  the  set 

S  =  {(x,y)|x2  +  y2-2x<0}, 

namely,  a  disc  of  radius  1  centered  at  (1,0). 

The  Hamiltonian  is 


3C  =  2x  -  2x2  -  2  y2  +  Xx  sin  cp -  X  cos  (p 
The  Hamiltonian  is  maximized  on  &■>  which  yields  the  optimal  controls 


sin  (p  = 


X. 


X,. 


\]^x  +X 


,  coscp  =  - 


yj^x+^y 


(18) 


Hence,  we  must  solve  the  TPBVP 

*  =  -f===f ,  *(°)  =  *o 


(19) 


y  = 


y(  0)  =  3^0 


Xx  =4x  — 2,  Xx(T)  =  0 


Xy=4y,  Xy(T)  =  0 


(20) 

(21) 

(22) 


III.3.1.  The  End  Game  in  the  Cartesian  Plane 

The  controls  don’t  directly  feature  in  the  cost  functional.  Hence  we  minimize  the 
derivative  of  the  integrand  in  the  cost  functional: 

d  /  2  2  \ 

—  x"  +  y  —  x  -»  mm 
dt 1  '  ’  <p 


2xsin<p-2ycos<^-sin<p  — >min 

<p 
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f 

X 

V 


V\ 

2, 


sin<^-  ycosp 


min 

<p 


sin  {(p\T)) 


1 

—  x 
2 

Jx2  +  y2  —x  + 


- -  9 

1 

4 


cos(^(r)) 


.V 


<Jx2  +  _y"  -x  + 


4 


Have  obtained  the  optimal  controls  in  the  end  game. 

•  In  the  end  game  E  heads  toward  the  BO  midpoint  M. 
Proposition  A  The  following  holds. 


U>) 

M>) 


=  const.  =  c,  0  <  t  <  T. 


Proposition  A 


(p  (t)  -  const. ,  0  <  t  <  T 


x{t )  =  xQ  +  sin  (p*  ■  t 
y(t)  =  y0  -  cos  (f)  -t 


(23) 


□ 


K  ~  4v0  +  4  sin  cp  ■  t  -  2 
A  =  4y0  -4cos^?  -t 

Ax{t)  =  2(2jc0  -l)t  +  2sin^>  -t2 -a 
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Ay  (7)  =  4y0t-2costp*  -t2  +b 


where  a  and  b  are  integration  constants. 
Furthermore,  Proposition  A  => 


(2x0  -l)f +  sin (p  ■ t 2 -a 
2y0t-cos(p  -r  +b 


V0<  t  <  T 


(sin<£>a  +  c-cos^*)f2  +(2.r0  -  l-2y0  -c)t  +  a-bc  =  0  VO  <  t  <  T 


<p  =  -  A  tan  c, 

1 

=  tan<£>  , 


Jo 

b=a- 


We  conclude  that  the  rElay  heads  in  a  straight  line  toward  the  midpoint  M. 
Finally,  given  the  initial  state  (x0,  y0),  we  calculate 


m  2.2  ,  A 

^max  ~JX0+  -V0  “  X0  +  ^  • 


Note  Since  Xx  (Tmiix )  =  0,  we  calculate  the  constant 

a=(\-2x())Tmm  -  sin (p  ■  T2m 


i  ro  i\t  cosg?  *  2 

b  =  (2x0-l)Tmm- - r  +  cos^  -7^ 

sin  (p 


Now 
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A  (T  )  =  2  2y()T  -cos cp* -T2  +  (2x,-l )T  +  cos  cp  -T2 

y  \  max  /  -'0  max  r  max  \  0  )  max  •  *  V  ma; 


max  •  * 

sin  cp 


=  4rmax  JO"  --*0  COt^ 
V  2. 


=  tan  cp 


A  (T  )  =  0. 

y  V  max  / 


and  the  “transversality  conditions”  Av(7max)  =  0,  /lv  ( 7nax )  =  0  hold. 

•  The  optimal  trajectories  are  straight  lines. 

III.3.2.  Value  Function 

The  optimal  trajectories  are 

x(r)  =  x  +  sirup*  •  t,  x(t )  =  ^ 

y(r)  =  y-cos(p"  ■  z,  y  (7)  =  y,  t  <  r  <  T 

and  the  (constant)  optimal  controls 


sin  cp  =  ■ 


cos  cp  =■ 


x2  +  y2  -  x  +  - 


lx2  +  y2  -x+ 


x2  (r)+  y2  (r)-x(r)  =  r2  +2  x —  sin  cp  -  y  cos  <p  ■z  +  x2  +  y2-. 

v  2 ) 


—  t  —  2.x  +  y  —  x-\ —  •  t  +  x  +  y  —  x 
V  '  4 
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Therefore,  the  value  function  is 


Hence,  have  obtained  the  explicit  value  function 

V(t,x,y;T)  =  2(T-t)-  x2  +  y2 -x-^x2  +  y2 -x  +  -^ -(T +  t)  +  ^(T2 +Tt  +  t2 
(x, y)eSr,  0 <t<T 

The  domain  of  definition  of  the  value  function  V  (/,  x,  y;  T  )  is  the  set 

S  =  j(x,y)|(x-l)2  +  );2<l,  x~  +y2>72| 

shown  in  Fig.  6 
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Figure  6:  The  domain  of  definition  of  the  value  function 


In  particular, 

V(x0,y0-T)  =  2T 


y0--x o' 


■yl 


-■T  +  -T2 
4  3 


(x0,>’0)g3V 


Note: 


r  i  2  ( 

When  T  =  Jx2  +  y2-x0  +  -  }/(x0,y0)  =  -1  ~2 


0+y0  2 


4  +  y~Q~  xo  +  ^ 


III.4.  Discussion 

This  is  an  interesting  optimal  control  problem:  The  end  state  is  free  and  the 
control  does  not  explicitly  feature  in  the  cost  functional,  which  requires  us  to  consider  the 
end  game.  Furthermore,  if  the  planning  horizon  T  is  long  enough,  all  the  optimal 
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trajectories  in  the  state  space  will  tenninate  at  the  “sweet  spot”  rE-  — ,  0-0.  At  the 


same  time,  to  solve  the  optimal  control  problem  we  employ  the  method  of  characteristics 
-  also  referred  to  as  Isaacs  method  [2],  which  requires  us  to  integrate  the  characteristics 
equations/state  and  co-state  equations  in  a  retrograde  manner,  "starting  out"  from  an  end 
state.  But  there  is  only  one  end  state,  the  critical  "sweet  spot".  Now,  recall  that  the 
optimal  control  provided  by  the  solution  of  the  end  game  does  not  apply  at  the  “sweet 
spot”  and  one  is  at  a  loss  of  how  to  kickstart  the  backward  integration.  This  indicates  a 
strong  singularity  at  this  critical  end  state,  the  "sweet  spot". 

Indeed  the  complete  optimal  flow  field/solution  "emanates"  from  the  special 
sweet  spot,  and  this  has  the  appearance  of  a  Big  Bang  type  situation.  It  would  appear  that 
since  a  family  of  characteristics,  and  not  just  a  single  characteristic,  emanates  from  the 
sweet  spot,  the  solution  of  the  characteristics  equation  is  not  unique,  and  Isaacs  method  is 
not  applicable  to  the  solution  of  the  rElay  optimal  control  problem.  This  difficulty  was 
overcome  by  slightly  backing  off  from  the  sweet  spot.  The  sweet  spot  was  replaced  by 
the  family  of  terminal  states  shown  in  Fig.  3,  namely  the  small  semicircle  centered  at  the 
sweet  spot  and  indicated  by  the  thick  line;  the  radius  of  the  semicircle  is  £  1. 

When  “starting”  the  integration  from,  say,  point  ET  on  the  circumference  of  the 
semicircle,  the  first  step  in  the  retrograde  integration  entails  the  use  of  the  optimal 
strategy  (14)  for  the  end  game;  the  end  states  ET  on  the  circumference  of  the  small  semi¬ 
circle  are  parameterized  by  the  angle  0  <  £  <  n.  This  kicks  off  the  integration,  and 
from  this  point  on,  one  uses  the  characteristics’  eqs.  (13).  The  integration  is  stopped  when 
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the  boundary  of  the  state  space  is  approached,  where  the  condition  rE  <  2 rQ  cos  0  is 
violated. 

III.5.  Differential  Game 

To  solve  the  differential  game  we  employ  the  method  of  characteristics,  a.k.a.,  the 
method  of  Isaacs.  The  partial  derivatives  of  the  value  function  are  related  to  the  costates 
according  to 


Vr  =-Ar  ,  Vr  =  —X  ,  V0=-Ao 

rE  rE  ro  ro  &  & 


Hence,  in  view  of  eqs.  (3),  (5),  (7)  and  (8),  the  equations  of  the  characteristics  are 


^-V  =  2r0  cos  0  -  Arr  -  ' 

at 


rE2y[vrp2rE+Vt2'  rE 


,  VAT)  =  0 


— V  =  2 rE  cos  8-2 rQ+a - .  =  = ,  V  (T)  =  0 

*  °  * 


~Ve  =  ~2rEr0  sin  6,  Vg(T)  =  0 
at 


d 


-  r,  = 


r  V 

rEV,F 


dt  '  /K/'i+V/ 


,  rE(T)  =  rE  ,rE  >0 


d  roYro  ^  - 

—  rn  =  a—  r0(T)  =  rQ  ,  rQ  >0 


dt 


4 


\\+v,2 


cW 

dt 


Va 


=  a 


'iT'TvT 


v"  o(j)=eT ,o<eT <cos_1(ii),  o<t<T 


2  rn 
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According  to  Isaac’s  method  [2],  the  characteristic  equations  are  integrated  in 
retrograde  fashion  “starting”  out  at  rE ,  r(h  ,  0T.  To  this  end  set  T  =  T  —  t.  Thus 


Now,  the  first  step  in  the  retrograde  integration  of  the  equations  of  the 
characteristics  requires  us  to  use  the  controls  from  the  end  game.  Thus,  the  first 
integration  step  is 


VrE  ( AT)  =  2AT {2rEr  -  r0r  cos  0T ) 
Vy0  (A?1)  =  2AT (r0T  -rEr  cos  0T ) 
Vg(AT)  =  2ATrEr0T  sin^r 
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Equations  (25)  are  obtained  from  eqs.  (24)  by  using  therein  the  terminal  optimal  controls 
derived  in  Section  2.1. 

Since  Vr)  (0)  =  0,  Vro  (0)  =  0  and  VH(0)  =  0,  the  first  step  (25)  is  required  and  the 
integration  of  the  equations  of  the  characteristics  (24)  starts  at  time  x=AT  with  the 
“initial”  condition (r£(AT),  r0(AT),  0(A T),  V%  (AT),  V,b  (AT),  Wg(A T)). 

The  following  holds. 

PROPOSITION  When  the  speed  ratio  0  <  a  <  2 ,  the  rElay  can  always  reach  the 
midpoint  M,  provided  that  the  game  horizon 


T> 


_\_ _ 

(2-a) 


4 rEr0  cos  6 


Proof 
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Figure  7 :  Pursuit  -  Evasion 


From  Fig.  7,  the  instantaneous  distance  EM  is 


-  4 rEr0  cos  9 


and 


d  =  — 
2d 


a 


2rEcos(p  +  -r0cosy/-r0cos9cos(p-arEcos9cosy/  +  rEr0  sin  6 


a  .  1  . 

—  sm^// - sin<p 

■  r  ** 

V  'o 


'E  J 

1  (a  3 

—  2 rE  cos <p-\ —  r0  cos y/ - r0  cos 9 cos cp - arE  cos 9 cos y/  +  arE  sin 9 sin y/ - r0  sin  9 sin cp 
2d\  2  ) 


if  oc 

—  <  (2 rE  -r0  cos^Jcos^-Tq  sin(9sin<p  +  — [(r0  -  2rE  cos9)cosy/  +  2rE  sin  9  sin 
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Now  min  max  d  yields  the  rElay  pursuit  strategy 

<P  ¥ 


rn  cos  6  -  2/y  .  rn  sin  6 

cos  <p  =  ,  ,  sin  (p  = 


yl*rE2 +r02 -4rErQ  cos  0 


^rE2  +ra2  -ArEra  cos  9 


The  rElay  heads  toward  the  midpoint  M  of  the  segment  BO  . 


The  motion  of  the  midpoint  M  is  exclusively  controlled  by  the  rOver  and  the  rOver’s 


evasion  strategy  is 


rn  -  2/y  cos  6  .  2/y  sin  9 

cos  y/  =  .  ,  sin  y/  = 


^4 rE2  +  rQ2  -  4 rEr0  cos  0 


V4v  +ro  cos 9 


Hence,  we  calculate 


d  =  — 
2d 


f 


'y/d te  +  ro  ~  4 rEr0  cos  9  +  —  oc^ArE  +  r02  -  4rEr0  cos  9 


=  ( «  -  2)  ,/4ry  +  r0'  -  4rcr0  cos  8 

=  i(a-2)<0 


and  therefore  the  time-to-go 


T(rE,  r0,  9)  =  d 
2 -a 


=  y/4r/  +  V  -  4rEr0  cos  0 

2- a 
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In  general,  optimal  trajectories  do  not  terminate  s.t.  a  configuration  where  B,  E 
and  O  in  Fig.  2  are  collinear  is  achieved.  If  however  the  game  horizon  T  is  sufficiently 

long  and  the  speed  ratio  parameters  0<  a  <  2,  at  some  point  in  time  a  state  s.t.  rE  =  ~ro 

and  0  =  0  will  be  reached  under  optimal  control,  after  which  the  game  is  rectilinear  and 

the  trajectory  arc  is  singular.  -  strictly  speaking  the  rElay  will  reduce  its  speed  to  —  a  and 


follow  the  rOver  s.t.  rE  (?) 


Hence  we  are  interested  in  the  trajectories  which 


“emanate”  in  retrograde  fashion  from  “sweet  spots”  ( rE  = 


0  =  0). 


It  is  however  impossible  to  kick  start  the  retrograde  integration  of  the  equation  of 
the  characteristics  using  eqs.  (25)  when  starting  out  from  a  “sweet  spot”.  Therefore, 
similar  to  the  optimal  control  problem,  we  "start  out"  from  the  family  of  "initial  states" 
parameterized  by  £,  as  shown  in  Fig.  3.  Thus,  in  eqs.  (25) 


rET  =  V  (  2  r°T  +  G  COS  ^ 2  +  ^  Sin2  ^  ’  rOr  =  rOT  ’  °T  =  A  tan 


^sin^ 

\r0r  +  s  cos  ^ 


where  0  <  s  1.  In  other  words,  in  eqs.  (25) 


=  t  T 'b.'  +  'bscosf +  £-,/•„  =r„  , 


#r(£;ror)  =  Atan 


^sin^ 


1 


r  +s  cos^ 


0  <^<n 
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III.6.  Optimal  Play 

In  Fig.  8  optimal  trajectories  in  the  state  space  are  shown. 


Figure  8:  Optimal  Relay  Trajectories  in  the  State  Space 

The  broken  lines  are  rElay  trajectories  in  the  state  space  which  terminate  at 
r0  =1  and  the  full  lines  are  their  projection  onto  the  half  disc  at  rQ  - 1 .  Optimal 

trajectories  which  terminate  at  ra  =1  are  constructed.  The  retrograde  integration  of  the 

equations  of  the  characteristics  yields  r£(r;£),  r0( V;£)  and  #(r;c).  rimx  is  determined 

by  the  condition  rE  (t^-,4)  =  2r0  (rmax;^)-cos(6'(rinax;^)),  that  is,  the  boundary  of  the 
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state  space  is  reached,  or,  0(rm!aL’,£)  =  0,  that  is,  the  plane  of  symmetry  of  the  state  space 
is  reached.  To  plot  the  trajectory  in  the  state  space,  are  calculated 


x(r;£)  =  r£(r;£)-cos6>(r;£), 
y(r;£)  =  r£(r;£)-sin6>(z-;£), 

ro(T>4) 


III. 6.1.  The  Optimal  Trajectories  in  the  Realistic  Space 

The  retrograde  integration  of  the  equations  of  the  characteristics  yields  the 
optimal  trajectories  rF  (r),  rQ(r),  6{x ),  0<T<Tmax. 


We  have  used  a  reduced  state  space  (r£,  rQ ,  0  )and  have  obtained  the  trajectory  and 
optimal  controls’  time  histories  in  a  rotating  frame  of  reference.  Specifically,  the 
sequences  rE^  =  rE(kAT),  r0(  =  r0(kAT),  6k  =0(kAT),  k=l,...,N  are  calculated,  where 

f  \ 


N  _  ^nax_  _ 

AT  0  2 


=  —  ^jl  +  4s  cos  ^  +  4s2 ,  rQ  =  r(k  =1,  0o-  A  tan 


^sin^ 


1 


+  scos^ 


In  addition,  the  optimal  control  sequences  (p*k  =  (p*  (kAT),  y/*k  =  y/*  (kAT), 
k=0,...,N-l  are  also  calculated.  To  better  visualize  the  optimal  trajectories,  they  are 
plotted  in  the  realistic  plane.  We  proceed  as  follows. 

Set 


E(kAT):=E((N-k)AT),  Q(kAT)  :=  0((N-k)AT),  k=0,...,N-l 
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and 


<p\kAT):=<p*((N-l-k)AT),  y/\kAT):=y/* ((N -l-k)AT),  k=0,...,N-l. 


Figure  9:  The  Construction  of  Optimal  Trajectories  in  The  Realistic  Space 


The  origin  of  the  realistic  plane  is  at  B  and  the  x-axis  of  the  realistic  plane  is  aligned  with 


the  segment  BO( 0).  Thus,  the  coordinates  of  the  rOver  and  rElay  at  time  t=0  are 


E (0)  =  (rE  (0)cos  (<9(0)) ,  rE  (0) sin  (<9(0))) 
O(0)  =  (ro(0),0) 
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cAt  time  t=0,  the  rElay  and  rOver  start  out  from 


00  =  O(O)  =  {r0  (0),  0)  and  E0  4  E(0)  =  (rE  (O)cos(0(O)),  rE( O)sin(0(O))) . 

Using  the  position  of  E  at  t=0,  E0  =  E{ 0 ) ,  and  the  optimal  control  of  E  at  time  t=0, 
qj  (0) ,  we  obtain  £j  =  E{AT )  as  shown  in  Fig.  9.  Likewise,  using  the  position  of  O  at 
t=0,  00=0{ 0),  and  the  optimal  control  of  O  at  time  t=0,  i//  (0),  we  obtain 
Ox  =  O(AT),  as  shown  in  Fig.  9.  Having  obtained  £j  =  E{AT),  construct  the  line  BEX 
and  use  the  rElay's  optimal  control  cp  (AT)  to  obtain  E2  =£’(2A7').  Similarly,  having 
obtained  (9,  =  0(A7’j,  construct  the  line  BOx  and  use  the  rOver's  optimal  control 
Y  {AT ^  to  obtain  02  =0{2AT). 

T 

This  process  terminates  at  time  N,  where  N  =  One  thus  obtains  the  trajectories 

Ek±(xE(kAT),  yE(kAT)  ),  and  Ok  ±(x0(kAT),  y0  {kAT )  ),  k  =  0,-,N. 

We  want  these  trajectories  to  be  presented  in  an  (x,  y)  frame  where  the  x-axis  is  aligned 
with  the  segment  BON. 

Having  obtained  0N  namely,  {x0  ,  y0_  j,  calculate  the  angle  rj  =  A  tan 

Note :  \lxoN2+yON2  =  ro  (N) = i 

To  get  the  rElay  and  rOver's  trajectories  in  the  new  (x,  y)  frame,  rotate  the  rElay  and 
rOver  trajectories  ?j  degrees,  namely 


Za 
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xE  :=  xE  cos  77  +  yE  sin  77 
yE  :=  ~xE  sin 77  +  yE  cost 7 


and 


x0  :=  x0  cos  r/  +  y0  sin  77 
y0  :=  ~xo  sin  77  +  ^0  cos  77 
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IV.  Analysis  and  Results 


IV.  1.  Optimal  Trajectories  of  Rover  and  Relay  for  different  speed  ratios  a ,  and 
angle  of  approach  of  “sweet  spot”  E, . 

The  cases  of  a  speed  ratio  a  =  0.5  and  angle  of  approach  E,  is  illustrated  in  Fig. 
10,  for  different  terminal  rOver  distances  from  the  base. 


Figure  10:  Optimal  trajectories  of  Rover  and  Relay  for  E,  =  10°,  «=0.5,  r0  =1,  2,  3 
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Figure  11:  Optimal  trajectories  of  Rover  and  Relay  for 

£  =  10°,  a  =  0.5,  1,  1.5,  r0j  =1 


Figs.  10  and  11  show  that  the  trajectories  don’t  scale  linearly  as  r()/  and  a  vary. 
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Figure  12:  Optimal  trajectories  of  Rover  and  Relay  as  E,  changes 
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Figure  13:  Optimal  trajectories  of  Rover  and  Relay  for  a  =  0.5,  q  =  0° 


When  E,  =  0°,  the  rElay  approaches  the  “sweet  spot”  at  an  angle  0  =  0°  during  the 
game.  The  rElay’s  displacement  is  twice  of  the  rOver’s  because  of  the  speed  ratio  a  =  —. 
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Figure  14:  Time  history  of  the  States,  Costates  and  Controls,  a  =  0.5,  c  =  0° 
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Figure  15:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  £  =  30°,  =  1 

Unlike  the  case  £  -  0° ,  when  £  -  30°  the  rOver  and  rElay  don’t  move  along  a 
straight  line.  However,  even  though  they  didn’t  stay  on  the  d  =  0  line,  eventually  they 
end  up  s.t.  <9  =  0.  The  broken  line  shows  the  end  position  at  rmax ,  where  the  three  points 
B,  E,  O  are  collinear. 
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Figure  16:  Time  history  of  the  States,  Costates  and  Controls,  a  =  0.5,  £  =  30°,  r0  =  1 


At  the  end  point  the  rElay’s  control  is  (p{ 0)  =  180°  -30°  =  150° 
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Figure  17:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  E,  =  120°,  rG  =  1 


When  E,  =  120°  the  rOver  and  rElay  start  the  game  at  a  non  zero  9  angle.  In  other 
words,  initially  the  base,  rOver  and  rElay  are  not  colinear.  The  optimal  trajectory 
“terminates”  on  the  boundary  of  the  state  space  s.t.  rE  (rmax)  =  2  r0  (^max)-cos(6l(rmax)). 
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Figure  18:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  0.5,  £  =  120°,  rOT  =1 

At  the  end  point  the  rElay’s  control  is  (p  (0)  =  180°  -120°  =  60° 
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- rOver 

- rElay 

*  Base 

Figure  19:  Optimal  trajectories  of  Rover  and  Relay  a  =  1,  £  =  0°,  rQ  =  1 


Unlike  in  the  previous  cases,  when  1  <  a  <  2 ,  the  rElay  and  rOver  start  and  end 
the  game  at  0  =  0,  irrespective  of  £  . 
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Figure  17:  Time  history  of  the  States,  Costates  and  Controls,  a  =  1,  =  0° ,  rG  =  1 


51 


rOver 
rElay 
*  Base 

0  0.2  0.4  0.6  0.8  1  1.2 


Figure  21:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  E,  =  180°,  rG  =  1 

The  rElay  and  rOver  should  start  and  end  on  a  line  since  E,  =  180°.  In  other  words, 
they  maintain  6  =  0.  However,  as  seen  in  Fig.  21,  the  simulation  didn’t  show  the  result 
we  expected.  This  is  because  of  a  small  numerical  error.  Without  the  error,  the  rOver  and 
rElay  each  will  start  the  game  at  0.5  and  0.75. 
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Figure  18:  Time  history  of  the  States,  Costates  and  Controls,  a  =  2,  %  =  180°,  rG  =  1 
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IV. 2.  Optimal  Trajectories  of  Rover  and  Relay  for  different  rE  rQ  0T 


- rElay 

O  rOver 
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1  3 

Figure  19:  Optimal  Trajectory  of  Relay  When  r£  =  — ,  r0  =—,a=  0,  0T-  20° 
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Figure  20:  Optimal  Trajectories  of  Rover  and  Relay  When 


rF  =-,  rf  =1,  a  =  1,  0T  =  1°,  2°,  20°,30°,40c 

^  UT  1 
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Figure  21:  Optimal  Trajectories  of  Rover  and  Relay  When 


rEj  =-,  r0r  =l,a  =  l,  0T=1°,  2°,  20°,  30°,  40° 


56 


Figure  22:  Optimal  Trajectories  of  Rover  and  Relay  When 


rE  -  — ,  r0  =  —,  a  =1,  0T  =  20° 

tT  ^  UT  ^  1 


Fig.  19  shows  the  case  when  the  rElay  doesn’t  have  enough  time  to  reach  to  the 
mid-  point.  This  is  the  optimal  control  case  when  the  rOver  is  stationary.  Therefore  the 
rElay  moves  straight  toward  the  mid-point,  however  unlike  in  our  previous  result,  it  stops 


moving  at  /; 


— .  Whereas,  the  rOver  is  moving  and  the  rElay  is  tracking  its  optimal 


position  continuously  in  fig.  22.  In  both  cases  the  rElay  was  not  able  to  reach  its  optimal 
position  but  it’s  following  the  guidance  of  the  optimal  control  law. 


From  these  results  now  we  know  we  can  check  if  the  rElay  is  optimally 
positioned.  And  if  it  is,  even  the  rElay  didn’t  arrive  its  optimal  point  we  are  able  to 
calculate  the  estimated  location  or  time  it  takes  until  the  rElay  will  reach  this  position. 
Moreover,  if  we  know  the  final  position  of  the  rOver  and  the  time  we  are  given,  even  if 
the  rElay  was  following  the  optimal  course,  we  can  find  the  final  optimal  point  of  the 
rElay.  Therefore  we  can  control  the  rElay  directly  to  the  intended  location. 

IV.3.  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 
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Figure  23:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

\  =  °-5>  roT  =  1>«  =  1>  £=0° 
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Figure  24:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

%  =  0-5,  rOT  =  1,  «  =  1,  £=30° 
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Ligure  25:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

rEr  =0.5,  t0t  =1,  a  =  1,  £=60° 
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Figure  26:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

^  =  °-5>  roT  =U  =  l  £=90° 
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Ligure  27:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

rEr  =0.5,  r0T  =  l,a  =  l,  £=120° 
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Figure  28:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

rET  =0-5’  roT  =la  =  h  £=150° 
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Ligure  29:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Control  Law 

rET  =0-5>  roT  =la  =  l  ^=180° 
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Figure  30:  Comparison  of  the  End  Game  Control  Law  and  Optimal  Relay  Control 

Law  =  0.5,  r0r  =  1,  a  -  0,  £=90° 

As  we  see  from  Figs.  23-30,  the  control  of  the  rElay  using  the  optimal  control  law 
and  the  end  game  control  law  look  very  similar.  However,  the  rOver’s  control  using  the 
end  game  control  law  is  quite  different  from  the  optimal  control  law. 

We  are  mostly  concerned  with  the  control  of  the  rElay.  Therefore  we  can  conclude  that 
the  end  game  rElay  control  law  is  a  good  sub-optimal  control  law. 
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V.  Conclusions  and  Recommendations 


V.l.  Conclusions 

This  thesis  develops  optimal  guidance  laws  for  a  rElay  MAV  in  support  of 
extended  range  ISR  operations.  The  algorithm  is  based  upon  the  solution  of  a  min-max 
optimization  problem,  namely,  the  solution  of  the  differential  game,  which  represents  a 
worst  case  scenario.  Heuristic  rElay  guidance  strategies  are  also  provided.  These  are 
derived  using  a  geometry  based  (sub)  optimality  principle,  and  also  the  solution  of  the 
one-sided  rElay  optimal  control  problem,  where  the  rOver  is  considered  stationary.  Both 
methods  provided  corroborating  results  which  were  then  employed  to  gain  insight  into 
the  solution  of  the  differential  game.  The  optimal  control  and  differential  game’s  solution 
exhibits  interesting  behavior:  the  optimal  flow  field  namely  the  optimal  trajectories  will 


converge  to  the  family  of  end  states  rE  =  —  rQ ,  where  from  the  optimal  trajectory  is  a 


singular  optimal  trajectory  and  r£(t)  =  — r0(t)  V  t.  A  parametric  investigation  was 


conducted  and  optimal  trajectories  where  generated  for  speed  ratio  parameters  0  <  a  <  2. 
When  the  speed  ratio  a  >  2,  a  “sweet  spot”  won’t  be  reached.  Obviously,  the  most 
interesting  case  is  a  =  1. 


V.2.  Recommendations  for  Future  Research 

The  interesting  cases  of 

1.  Higher-order-dynamics  of  the  rElay  MAV  must  be  modeled,  including  a  time 
delay  in  the  control  loop. 
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2.  Multi  player  scenarios  must  be  addressed:  One  rElay  MAY,  multiple  rOver 


MAVs  needs  to  be  considered. 

3.  Multiple  relay  MAVs  and  multiple  rOver  MAV  scenarios  must  be  considered. 

4.  Investigate  different  RF  power  requirements  for  Rover  and  Relay 
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Appendix  A  -  Additional  Scenarios 


Figure  A.  1:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  £  =  60° ,  rG  =  1 
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q[ - 

Figure  A.  2:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  0.5,  <^  =  60° ,  yQt  =  1 


At  the  end  point  the  rElay’s  control  is  cp  (0)  =  180°  -60°  =  120° 
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Figure  A.  3:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  £  =  90°,  rG  =  1 
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Figure  A.  4:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  0.5,  B,  =  90° ,  r0j  =1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  180°  -90°  =  90° 
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Figure  A.  5:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  =  150°,  rG  =  1 
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Figure  A.  6:  Time  history  of  the  States,  Costates  and  Controls 

a  =  0.5,  £  =  150°,  r0r  =1 

At  the  end  point  the  rElay’s  control  is  (p\ 0)  =  180°  -150°  =  30° 
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Figure  A.  7:  Optimal  trajectories  of  Rover  and  Relay,  a  =  0.5,  £  =  180°,  r(?  =  1 
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Figure  A.  8:  Time  history  of  the  States,  Costates  and  Controls 

a  =  0.5,  £  =  180°,  r0j  =1 
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Figure  A.  9:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1,  £,  =  30°,  r0  =  1 
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Figure  A.  10:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  l,  £  =  30°,  r0r=l 

At  the  end  point  the  rElay’s  control  is  (p*  { 0)  =  180°  -30°  =  150° 
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Figure  A.  11:  Optimal  trajectories  of  Rover  and  Relay.  a  =  1,  £  =  60°,  =  1 
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Figure  A.  12:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  l  £  =  60°,  rOT=l 


At  the  end  point  the  rElay’s  control  is  (p( 0)  =  180°  -60°  =  120° 
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Figure  A.  13:  Optimal  trajectories  of  Rover  and  Relay.  a  =  1,  %  =  90°,  rG  =  1 
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Figure  A.  14:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  l,  £  =  90°,  r0r  -1 

At  the  end  point  the  rElay’s  control  is  (p  (0)  =  180°  -90°  =  90° 
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Figure  A.  15:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1,  <2  =  120°,  rG  =  1 


Figure  A.  16:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  l,  £  =  120°,  rQr=  1 

At  the  end  point  the  rElay’s  control  is  (p  (0)  =  180°  -120°  =  60° 
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Figure  A.  17:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1,  £  =  150°,  rG  =  1 
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Figure  A.  18:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  l,  £  =  150°,  r0T=  1 


At  the  end  point  the  rElay’s  control  is  (p( 0)  =  180°  -150°  =  30° 
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Figure  A.  19:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1,  £  =  180°,  rG  =  1 
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Figure  A.  20:  Time  history  of  the  States,  Costates  and  Controls. 

cc  =  l  #  =  180°,  r0r=  1 


At  the  end  point  the  rElay’s  control  is  (p  { 0)  =  180°  - 180°  =  0° 
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Figure  A.  21:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £,  =  0°,  rG  =  1 
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Figure  A.  22:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  1.5,  £  =  0°,  r0r=l 
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Figure  A.  23:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £  =  30°,  r0  =  1 
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Figure  A.  24:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  1.5,  £  =  30°,  r0i  =1 


At  the  end  point  the  rElay’s  control  is  <p\ 0)  =  180°  -30°  =  150° 
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Figure  A.  25:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £  =  60°,  r0  =  1 
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Figure  A.  26:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  1.5,  £  =  60°,  rOT  =1 


At  the  end  point  the  rElay’s  control  is  cp  (0)  =  120° 
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Figure  A.  27:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £  =  90°,  =  1 
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Figure  A.  28:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  1.5,  £  =  90°,  r0r=l 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  90° 
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Figure  A.  29:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £,=  120°,  rQ  =  1 
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Figure  A.  30:  Time  history  of  the  States,  Costates  and  Controls 

a  =1.5,  %  =  120°,  rOT  =1 


At  the  end  point  the  rElay’s  control  is  (p\ 0)  =  60° 
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Figure  A.  31:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £  =  150°,  rG  =  1 
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Figure  A.  32:  Time  history  of  the  States,  Costates  and  Controls 

«  =1.5,  £  =  150°,  rOT  =1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  30° 
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Figure  A.  33:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.5,  £,  =  180°,  rG  =  1 
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Figure  A.  34:  Time  history  of  the  States,  Costates  and  Controls 

«  =1.5,  £  =  180°,  rOT  =1 
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Figure  A.  35:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  £  =  0°,  r0  =  1 
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Figure  A.  36:  Time  history  of  the  States,  Costates  and  Controls. 


a  =  1.9,  £  =  0°,  ra  =1 
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Figure  A.  37:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  E,  =  30°,  rG  =  1 
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Figure  A.  38:  Time  history  of  the  States,  Costates  and  Controls 

a  =  1.9,  #  =  30°,  r0j=  1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  150° 
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Figure  A.  39:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  £  =  60°,  =  1 
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_ _ _ 

- — "" 

\ 

1 

Figure  A.  40:  Time  history  of  States,  Costates  and  Controls,  a  =  1.9,  £,  =  60°,  rG  =  1 


At  the  end  point  the  rElay’s  control  is  cp\ 0)  =  120° 
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Figure  A.  41:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  £  =  90°,  =  1 
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Figure  A.  42:  Time  history  of  the  States,  Costates  and  Controls 

a  =  1.9,  £  =  90°,  r0r=  1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  90° 
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Figure  A.  43:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  <2  =  120°,  r0  =  1 
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Figure  A.  44:  Time  history  of  the  States,  Costates  and  Controls 

a  =1.9,  £  =  120°,  r0r=  1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  60° 
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Figure  A.  45:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  t,  =  150°,  r0  =  1 
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Figure  A.  46:  Time  history  of  the  States,  Costates  and  Controls 

a  =1.9,  £  =  150°,  r0r  =1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  30° 
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- rOver 

- rElay 

*  Base 

Figure  A.  47:  Optimal  trajectories  of  Rover  and  Relay,  a  =  1.9,  £,  =180°,  rG  =  1 
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Figure  A.  48:  Time  history  of  the  States,  Costates  and  Controls 

a  =  1.9,  g  =  180°,  rOT  =1 
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- rOver 

- rElay 

*  Base 

< - 

Figure  A.  49:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  £  =  0°,  r()/  =  1 
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Figure  A.  50:  Time  history  of  the  States,  Costates  and  Controls. 


a  =  2, 


£  =  0°, 


r 

'  Or 


=  1 
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Figure  A.  51:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  £  =  30°,  rG  =  1 


115 


x  10 


tau  tau  tau 


Figure  A.  52:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  2,  £  =  30°,  rOT=l 
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At  the  end  point  the  rElay’s  control  is  cp  (0)  =  150‘ 
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Figure  A.  53:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  £  =  60°,  r0  =  1 


Figure  A.  54:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  2,  £  =  60°,  r0r  =1 


At  the  end  point  the  rElay’s  control  is  cp  (0)  =  120° 
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Figure  A.  55:  Optimal  trajectories  of  Rover  and  Relay,  a-  2,  £  =  90°,  rG  =  1 
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Figure  A.  56:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  2,  £;  =  90°,  r0r  =1 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  90° 
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Figure  A.  57:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  £  =  120°,  rG  =  1 


Figure  A.  58:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  2,  £  =  120°,  rGr=l 


At  the  end  point  the  rElay’s  control  is  <p\ 0)  =  60° 
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Figure  A.  59:  Optimal  trajectories  of  Rover  and  Relay,  a  =  2,  %  =  150°,  rG  =  1 
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Figure  A.  60:  Time  history  of  the  States,  Costates  and  Controls. 

a  =  2,  £  =  150°,  rGr=l 


At  the  end  point  the  rElay’s  control  is  (p  (0)  =  30° 
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Appendix  B  -  Geometry 


An  Elementary  Euclidean  Geometry  Result: 

It  is  well  known  that  the  locus  of  all  points  such  that  the  sum  of  the  distances  from 
two  fixed  points  is  constant,  is  an  ellipse.  Thus,  the  following  is  of  some  interest. 

Theorem  1  The  Locus  of  all  points  such  that  the  sum  of  the  squares  of  the 
distances  from  two  fixed  points  is  constant,  is  a  circle  centered  at  the  midpoint  of  the 
segment  formed  by  the  two  fixed  points.  The  radius  of  this  circle  is 

R  =  4d2~f2 

where  the  sum  of  the  squares  of  the  distances  is  2d"  and  the  distance  between  the  fixed 
points  is  2/ ;  obviously,  d>  f  . 

Proof: 

Let  the  fixed  points  F\  and  F2  be  on  the  x-axis  (F\  =  (f,  0),  F2  =  (-/,  0))  as  shown 
in  the  figure  below. 
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y 


Figure  B.  1:  Schematic  of  Fixed  Points  Showing  Isocost  Circle 


The  sum  of  the  squares  of  the  distances  is  calculated  as 

2  d2  =  (f  +  x)2  +  y2  +  (f-x)2  +  y2 
=  2/ 2  +  2x2  +  2y2 
=>  x2  +  y2=d2-f 2 

This  is  the  equation  of  a  circle  centered  at  the  origin,  whose  radius  is 

R  =  yjd2-f2 

□ 


This  result  appeared  in  [5]. 

Remark:  The  loci  of  constant  costs,  2 dr,  are  concentric  circles  where  the  minimum  cost 
is  found  at  the  midpoint  of  the  line  formed  by  F\  and  Fi,  where  d  =/. 

Extension:  The  Locus  of  all  points  such  that  the  weighted  sum  of  the  squares  of  the 
distances  from  two  fixed  points  is  constant,  is  a  circle  centered  on  the  segment  formed  by 
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the  two  fixed  points  and  is  at  a  distance  of  (1  -  2 a)f  from  this  segment’s  midpoint.  The 
radius  of  this  circle  is 

R  = 

2 

where  d  is  the  specified  weighted  sum  of  the  squares  of  the  distances,  the  distance 
between  the  fixed  points  is  2/;  and  the  weight  is  a\  if  a  <  0  or  a  >  1  this  is  true  Md  >  0 , 

and  if  0  <  «  <  I ,  d  >  2f^ja(\  —  a)  .  Note:  When  the  weight  a  =  Vi,  need  d  >  f  . 

Proof:  The  weighted  sum  of  the  squares  of  the  distances  is  calculated  as 

d2  =a\_(f  +  x)2  +  y2~\  +  (l-a)\_(f -x)2  +  /] 

=  af 2  +ax2  +  2  afx  +  ay2  +(1  -a)f2  +(1  -a)x2  -2(1- a)fx  +  (1- a)y2 
=  f2  +  x2  +  y2  -  2fx(l  -  2a) 

=  \x-(l-2a)f]2+f2  +  y2-(l-2a)2f2 

=>  [x-(l-2a)ff  +  y2  =d2 -4a(l-a)f2 

□ 
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Appendix  C  -  New  Parameterization  of  Family  of  “Endpoints” 


Each  “sweet  spot”  on  the  line  rE  =  —  r0  is  encapsulated  in  a  small  hemispherical 
terminal  manifold,  as  shown  in  Fig.C.l 


0 


0T  :=  s  sin ^ 


cosgcost] 


rET  -=  -ror  COS^Simj 

0  <  rj  <  In 


0<£  <- 

2 


Figure  C.  1:  End  States’  Manifold 
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rs=2ro 


r  ,  r  >  0 . 

Oj  7  Oj 


As  in  the  optimal  control  case,  we  back  off  from  “sweet  spots”  and  the  retrograde 
integration  of  the  characteristics  equations  is  initiated  from  end  states  ( rFr ,  r  ? ,  dT  j  on  a 
hemispherical  manifold,  as  shown  in  Fig.C.l,  where 

0T-s  sinE, 


roT  '■=  r„T  cosE  COST] 
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rET  :=  ~  ''n,  ~£  COS4  SUV1 


0  <  77  <  2n 


0<^§ 


The  end  states  associated  with  the  “sweet  spot”  —  r  ,  r  ,  0  reside  on  a  hemispherical 

v  2  ' 


terminal  manifold,  as  shown  in  Fig.C.2,  where  the  state  space  of  the  differential  game  is 
illustrated. 


Without  loss  of  generality  we  confine  our  attention  to  ro  =  1 .  The  family  of  optimal 


trajectories  is  thus  parameterized  by  0  <  E,  <  —  and  0  <  77  <  2k  .  The  retrograde 


integration  stops  when  rE  =  2rocos6 . 


The  optimal  control  of  the  rElay  at  t=T  is  given  by  the  solution  of  the  end  game 
where  the  end  state  is  rF  ,r  ,0T.  It  is 


rOT  COS 

l4rET2  +rOT2  ~4rETroTCOS0T 


dn(^(r)): 


_ rOT  sin  eT _ 

)4rET  +  roT  ~4rErrOp  COS$r 


provided  that  the  end  state  is  not  rE/  ~  —  r0/  ,  6r  =  0. 
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Similarly,  the  optimal  control  of  the  rOver  at  t=T  given  by  the  solution  of  the  end 


game  is 


cos(i//(r)) 

sin(y/(r)) 


roT  -%  cos  0T 


V 


%  +r0T--2rETr0TC°S&T 


rA;  sin  6t 

rEr  r0T  —  2rET  r0T  COS 


(27) 


provided  that  the  end  state  is  not  ,  0T  =  0 ;  when  this  is  the  case,  from  first 

principles  we  conclude  that  the  rOver’s  control  y/*  (T)  =  0. 

Similar  to  the  solution  of  the  optimal  control  problem,  the  “sweet  spot”  is 


terminal  controls  given  by  eqs.  (26)  and  (27)  cannot  be  computed.  Therefore,  similar  to 
the  optimal  control  problem,  one  backs  off  and  one  considers  the  terminal  states  on  the 
hemisphere  shown  in  Fig.C.l. 

The  end  states  on  the  small  hemisphere  around  the  “sweet  spot”  are  parameterized 
as  follows. 
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0T  :=  s  sin  E, 

roT  '■ =roT  -^COS^  COS  7) 

r et  •—  ~  r0r  -^cos#  sin/7 
0  <  77  <  In 

os«<! 

0  <  s  =  1,  fixed 


(28) 


The  optimal  controls  at  t=T  for  end  states  on  the  hemispherical  terminal  manifold  are 
obtained  by  inserting  eqs.  (28)  into  the  terminal  control  eqs.  (26)  and  (27). 

We  calculate 


4rET  “  +  V  “  4rET  W  COS  °T~  4 


^  r0/ '  +  £~  cos2  E,  sin2/7  -  sr^  cos  E,  sin 77 


+  /;  2  +  £2  cos2  E,  cos2  77  -  2.st  cos  cos  77 

r  1  ^  ^ 

2 


-4 


—  7^  —  cos  ^  sin 77  -fcos^  COS77)  1-  —  ^sin2^ 


■  s'2  /;r2  sin2  <^  +  3cos2  sin277  +cos2  <^-2cos2  E,  sin2 77^ 


roTsineT  ~£rOTsinZ; 

r0j  cos  6j  -  2  rEj  «  £  cos  ^  ( 2sin  77  -  cos  77 ) 


Inserting  these  expressions  into  the  rElay’s  tenninal  control  eqs.  (26)  yields 


cos  (49*  (r)) 

sin  (V  (t)) 


cos£(2sin77-cos/7) 

Jrh2  sin2  £  +  3cos2  E,  sin2/7  +  cos2  £-2  cos2  E,  sin2/7 

_ >'oT  sin  4 _ 

Jro2  sin2  £  +  3  cos2  E,  sin2/7  +  cos2  £-2  cos2  E,  sin2/7 


Next, 
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1 


rE  1  +  ro  1  ~  2 rE  ro  cos  ~  ~  ro  2  +  £ 2  cos"  £  sin"/;  -  fr0  cos  £  sin/;  +  ra  z  +  sL  cosz  £  cosz  /; 


2  ,  2 


-  r  cos  £  cos  r/ 


-2 


-r  -£cos^  sin/;  (rr -£cos^  cos/;)  1-  — £-2sin2^ 
1  7V  2 


=  ^roT2  +  sroT  cos  ^  (sin/;  -  cos  /;) 


+  £• 

1 

« —  r 

4  t 


^roT2  sin2  £  +  cos2  £-2cos2  £  sin/;  cos/; 


ror -rErcosOT  =^rOT  +  £cos£(sin/;-cos/;)  +  -U2r0j,  sin2  £  *  ^-r0r 

(l  3  1 

r£  sin  ^  -s  cos  £  sin/;  sin  ^  «  —sr,  sin£ 

v  2  T  /  2  T 


— »  cos 


(y/(r))*l,  sin(^*(r))*0 


Inserting  these  expressions  into  the  rOver’s  terminal  control  eqs.  (27)  yields 


Using  these  controls  at  time  t=T,  we  obtain 


rE  = 


cos  ^{isvari -cos/;) 


^jra/  2  sin2  4  +3cos2  4  sin2/;  +  cos2  3;  -2  cos2  4  s i n 2 /; 


,  r0=  a 


^  sa  sin  {(^-2  S  cos^sin  /;) 


V 

' 0T 


7,,  «in  £ 


(1 


r  -< 


•sxos^sin/;  ^Jr^2  sin2  +  3 cos2  £;  sin2/;  +  cos2  -2 cos2  sin2/; 
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We  note  that  T  if  0  <  ,  <  *  and  rE  l  if  *  <  ,  <  * .  that  is,  t  if  ,,  <  ^  and  4-  if 


rE  >  ,  6  <  0.  Hence,  the  trajectory  heads  toward  the  locus  of  “sweet  spots’ 


s.t.  rQ  t . 


The  above  differential  equations  allow  us  to  propagate  the  state  to  time  T  -  AT  : 


:(r-Ar)  =  ^r0r-£cos£sin/7 


cos£(2sin/7-cos/7) 


^?;r2sin2£  +  3cos2£  sin2?7  +  cos2£-2cos2£  sin2 77 


■AT 


,(T -AT)  =  rOT  -scos%  cosr/-a-AT 


6(T-AT)  =  £sm%  + 


ra,  sin  £ 


sin 


-7;  -fcos^sin/7  J70  2sin2£  +  3cos2£  sin277  +  cos2£-2cos2£  sin2 77 
v2 

'm%(roT  -2fcos^  sin 77) 


■AT 


■■AT 


To  propagate  the  characteristics/co-states  to  time T- AT (r  =  AT),  we  proceed  as 


follows. 


Recall 
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—  V  =  4rF  -  2rn  cos  6  + 

dr  rE  e  0 


d 


>i\/'y X +v> 


v ; 


9 


-V.  =  2rn-2rFcos6-a  r 


•  n,(o)=o 


o  V'O  1  'e 


—  =  2r£r0  sin  0,  (0)  =  0,  0<z  <T 

dz 


Now,  on  the  one  hand, 


sin  cp  = 


yj^+V,2 


and  on  the  other  hand,  we  have  calculated  the  terminal  rElay’s  control 


sin  [(p  T{0T,  r0, ,  rFi  ]) 


ro  sin^ 


A^ro  2sin24r  +  3cos24:  shr^  +  cos^^cos2^  sin2^ 


Similarly,  on  the  one  hand 


sin^  = 


V, 


Jdvd+v/ 


and  on  the  other  hand,  we  have  calculated  the  rOver’s  tenninal  control 


sin  (y/\[0T,  r0r, 


=  0 


Thus,  the  limits 


lim 


Va 


4 


lim 


Va 


Vg.V^^co  12  y  2+y2  Vg,  vrlr->oo  lr  2V  2_j_t/2 


E  r  rF  1  r9 


4 


rlv  +V 

'O  V  r,  ^  V  a 


exist. 
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at  r=0 


hX,2+v,2 


r0\2+v; 


J^v„=2(2rB-r0  cos#),  F_(0)  =  0 
j-Vri>=2(2r0-rEcos#),  Vj0)  =  0 


X,(AT) 


(  1  | 

=  2  r  -2fcos<^  sin/7-  r,r -  — r  f2sin2^-^cos^  COS/7  +  — £3cos^  sin2 1;  cos 77 

f  ]  1  n  T  ^ 

=  2sAT  —ros sin2  ^-2cos^  sin/7  +  cos ^  cos 77  —  £2cos<^  sin2  cos/7 

,  2  r  2  y 


n,  (Ar) 


/  1  1  1 

=  2  r  - ^ cos ^  cos 77 -  —  r0y -  —  r^f2 sin2 cos ^  sin/7 +  —  £'3 cos ^  sin2^  sin/7 


=  Ar  r  -2fcos^  COS/7  + —  r  ^2  sin2  <^  +  2^cos^  sin/7  +  ^3 cos^  sin2  ^  sin/7 

,  r  2  r 


MAO 


1  ,  1 


=  2^sin4Ar  —rOT"-—erOT  cos£  cos/7  -  r  ^cos^  sin/7  +  ,?- cos<^  sin/7  cos/7 


From  this  point  on,  the  retrograde  integration  of  the  characteristics’  eqs.(10)  is 


undertaken. 


136 


We  show  the  family  of  optimal  trajectories  in  an  (x,y)  plane  where 


x(t)  =  rF(t)cos(0(t)),  y(t)=r E(t)  sin( 0(t)) . 
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Appendix  D  -  Suboptimal  Solution 


Suboptimal  solutions  are  useful  in  their  own  right  and  provide  insight  into  the 
optimal  control  problem  and  differential  game 

Geometric  Approach 

Using  a  geometric  approach  provides  a  suboptimal  but  easily  implementable 
solution  of  the  differential  game.  This  approach  is  suboptimal  because  the  rElay  and  the 
rOver  each  momentarily  assume  that  the  other  player  is  stationary  when  determining  their 
optimal  control. 


B 


Figure  D.  1:  Schematic  of  Relay  System  Showing  the  Midpoint 


The  geometry  of  the  engagement  forms  a  triangle  with  vertices  E,  B  and  O 
representing  the  respective  locations  of  the  rElay,  Base  and  rOver  (see  Figure  D.l).  Let  M 
be  the  midpoint  between  the  rOver  and  the  Base.  Simply  rotating  the  schematic  in  Figure 
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D.l  provides  an  equivalent  schematic  (see  Figure  D.2)  which  is  similar  to  the  one 


analyzed  in  Appendix  A. 


E 


Figure  D.  2:  Schematic  of  Relay  System  Showing  Isocost  Circle 


If  the  rOver  were  stationary,  the  loci  of  constant  instantaneous  costs 

y  =  EO  +  BE 

for  the  rElay  are  concentric  circles  centered  at  the  midpoint  and  the  midpoint  is  the  rElay 
location  which  minimizes  the  cost. [2]  The  rElay  is  on  the  circumference  of  said  circles, 
and  the  instantaneous  cost  y  is  determined  by  the  position  of  the  rElay.  This  means  that 

the  gradient  vector  for  minimizing  cost  is  in  the  radial  direction.  Therefore,  the  optimal 
strategy  of  the  rElay  is  to  head  toward  the  midpoint  M. 

The  optimal  control  of  the  rElay  is  determined  using  the  triangle  ABEM.  The 
distance  between  E  and  M  is  determined  using  the  law  of  cosines  (just  as  in  determining 
the  distance  between  E  and  O  before).  The  control  angle  (p  is  then  found  indirectly  by 
finding  its  supplementary  angle  using  the  law  of  sines.  However,  due  to  an  inherent 
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ambiguity  in  the  law  of  sines,  the  control  law  is  specified  for  three  cases:  (1)  (p  is  acute, 


(2)  (p  is  90°  and  (3)  cp  is  obtuse. 


qr  = 


sin 

K 

;r-shT 


r0  sin  9 


^rE+r^-4rEr0cos9 


r0  sin  9 

y4ArE+ro-ArEro  cos  9 


..  rn  cos  9 

for  r,  <  — - 

£  2 

rn  cos  9 


for  rE  = 


for  rE  > 


2 

rn  cos  9 


(29) 


This  ambiguity  can  be  bypassed  by  using  an  inverse  cosine  function  in  place  of  the 
inverse  sine,  i.e. 


q>*  =  cos 


f  \ 

r0  cos9-2rE 

v  \l^rE  +ro  ~  4r£ro  cos  9  ^ 


(30) 


y/*  =  <p*-9 


(31) 


Note  that  these  rElay  strategies  are  independent  of  the  planning  horizon  T. 

Once  E ,  B  and  O  are  collinear,  reducing  the  rElay  velocity  eliminates  the  need  for 
excessive  control  use.  However,  the  rElay  might  never  actually  arrive  at  the  midpoint  due 
to  a  short  planning  horizon  T,  or  the  maximizing  efforts  of  a  fast  rOver.  If  the  rOver  used 
a  suboptimal  control  strategy  (which  would  usually  be  the  case  in  practice),  the  rElay 
may  be  able  to  always  arrive  at  the  midpoint  and  consistently  match  the  motion  of  the 
midpoint;  this  is  a  singular  trajectory. 
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An  additional  one-sided  optimization  problem  is  obtained  when  the  rOver’s  point 
of  view  is  taken,  namely,  the  rElay  is  stationary  and  the  rOver  works  to  maximize  the 
cost  functional. 

y  =  £  ( 2 4  +  rl  -  2 rEr0  cos  9]  dt 
The  rOver  will  run  away  from  the  rElay. 

Numerical  Results 

Guided  by  the  suboptimal  solution  and  the  solution  of  the  one  -  sided  optimal 
control  problem,  the  differential  game  is  solved  using  Isaacs  method  [2],  namely,  the 
retrograde  integration  of  the  characteristics’  equations  (10).  In  the  figures  below,  the 
spatial  results  are  shown.  The  following  numerical  results  show  the  solution  of  the 
differential  game  where  T  =  0.25,  a  =  1,  i)  =  .5,  /;,  =  1  and  00=  f . 
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Spatial  Results 
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Figure  D.  3:  Relative  Spatial  Results  for  T  =  .25,  a  =  1,  =  .5,  r() o  =  1  and  0O  =  f 


The  following  numerical  results  show  the  solution  of  the  min-max  problem  where 
T  =  .49,  a- 1,  rQa  =1  and  0o  =  f. 
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Spatial  Results 
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Figure  D.  4:  Relative  Spatial  Results  for  7  =  1,  a  =  1,  ri:  =  .5,  rQ  =  1  and  8t)  =  ^ 
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